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OFFICE  OF  THE  DIRECTOR  OF 
DEFENSE  RESEARCH  AND  ENGINEERING 
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INTRODUCTION 


A  topical  area  review  of  research  and  development  on  testing  and  on  plans  for  application 
of  the  findings  to  the  selection  and  classification  of  enlisted  personnel  was  held  on  December  8-9, 
1988,  at  the  Institute  for  Defense  Analyses  (IDA),  Alexandria,  Virginia.  The  purpose  of  the 
review  was  to  bring  together  the  research  and  development  and  the  user  communities  concerned 
with  testing  to  ensure  that  each  is  aware  of  the  other's  activities  and  that  the  research  and 
development  programs  meet  the  needs  of  the  existing  and  planned  selection  and  classification 
programs. 

The  review  was  held  at  the  request  of  George  P.  Millbum,  Deputy  Director,  Defense 
Research  and  Engineering  (Research  and  Advanced  Technology)  and  Grant  S.  Green,  Jr., 
Assistant  Secretary  of  Defense  (Force  Management  and  Personnel).  They  were  represented  at  the 
review  by  Dr.  Earl  A.  Alluisi,  Assistant  for  Training  and  Personnel  Systems  Technology 
(ODDR&E/R&AT)  and  Dr.  Wayne  S.  Sellman,  Director  for  Accession  Policy  (OASD/FM&P), 
who  are  responsible,  respectively,  for  research  and  development  on  testing  and  for  the  application 
of  research  on  testing  to  selection  and  classification. 

Interest  and  concern  have  been  expressed  by  the  Congress,  the  Office  of  the  Secretary  of 
Defense,  and  the  Services  regarding  the  R&D  test-development  process  to  ensure  equitable 
screening,  selection,  and  classification  of  enlisted  personnel.  A  major  purpose  of  the  review, 
therefore,  was  to  discuss  plans  for  the  implementation  of  the  products  being  developed  by  the 
R&D  program.  Additionally,  the  review  provided  an  opportunity  to  identify  any  gaps  in  R&D  that 
may  need  to  be  filled,  as  well  as  any  new  developments  or  initiatives  for  which  transition  plans 
need  to  be  drawn. 

Presentations  were  made  by  members  of  the  Service  personnel  research  laboratories, 
Service  headquarters  and  Defense  agencies.  Attendees  included  additional  members  of  these 
organizations  as  well  as  representatives  of  the  using  agencies  that  test,  select  and  classify  members 
of  the  military  services.  Material  presented  at  the  meeting  is  contained  in  this  report. 


In  addition,  a  White  Paper  on  Enlisted  Testing  will  be  prepared  and  published  later  to 
provide  a  summary  technical  review  of  the  military  value  of  testing  and  the  potential  for  increased 
value  with  further  development  and  implementation  of  the  area. 


Earl  A.  Alluisi 

Office  of  the  Deputy  Director  of 
Defense  Research  and  Engineering 
(Research  and  Advanced  Technology) 


Wayne  S.  Sellman 

Director  of  Accession  Policy 

Office  of  Assistant  Secretary  of  Defense 

(Force  Management  and  Personnel) 
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THE  ARMY  PROGRAM 


N.  KENT  EATON 
ARMY  RESEARCH  INSTITUTE 
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TOPICAL  AREA  REVIEW  —  TESTING  R&D 
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U.S.  Army  Research  Institute 

Selection  &  Classification  Technical  Area 
AV  284-8275  or  Comm  (202)  274-8275 


ARI  ORGANIZATIONAL  STRUCTURE 
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FIELD  UNITS  AND  COORDINATION/ 
RESEARCH  ELEMENTS 


THE  ARI  WORK  PROGRAM 
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$25.3M  BUDGET  $59.7M  BUDGET  SYSTEMS  RESEARCH  LABORATORY 
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ARI  Selection  and  Classification  Technical  Area 


Developing  methods  for  setting  minimum 
qualifying  standards  for  new  MOS 

I.  Implementing  newly  developed  tests 

i.  Determining  the  value  of  alternative 
selection  systems 


Selection  and  Classification  Research 

Topical  Area  Review 


13 


Clinton  Walker 


DEVELOPING  AND  VALIDATING  ARMY 
SELECTION  AND  CLASSIFICATION  TESTS 


DARLENE  OLSON 
ARMY  RESEARCH  INSTITUTE 
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RESEARCH  TASK 
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-  Temperament 

-  Vocational  Interest 

Develop  Algorithms  and  Computer-Based  Procedures  to 
Improve  (Optimize)  Soldier- MOS  Assignment 


PROJECT  A  PREDICTORS 


PROJECT  A  PERFORMANCE  MEASURES 
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Project  A 

Gains  in  Predicting  First  Tour  Performance 
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•ABLE  Predicts 


UCOFT  Gunnery  Performance 
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Project  A  Test  Improves  Live  Fire  Hits 
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Source:  ARI  November  1988  ‘Target  2000  Yards 
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PREDICTION  OF  JOB  PERFORMANCE 

USING  THE  ABLE 


ABLE  Adjustment  Score  — ►  Attrition 
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ARMY 

OVERALL  SATISFACTION 
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To  identify  minimally  qualified  recruits 
To  identify  recruits  with  outstanding 
potential 


SYNTHETIC  VALIDATION  PROJECT 
RESEARCH  DESIGN 


Three  job  component  models 
Three  approaches  to  setting 
performance  standards 


Identification  of  Prediction  Equations 
Using  Synthetic  Validation  Techniques 
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SYNTHETIC  VALIDATION  RESEARCH: 
PROGRESS  TO  DATE 
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Comparison  of  Different  Kinds 
of  Judges 


SINGLE-RATER  RELIABILITY  OF  JOB 


Based  on  Core  Technical  validity  rating 


CORRELATION  OF  MEAN  JOB  DESCRIPTION  PROFILES 
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and  Attrition 


STANDARD  SETTING  RESEARCH: 
PROGRESS  TO  DATE 
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Tryout  on  Three  MOS 


(0 

UJ 

Sj 

-J 

Ui 

o 


oc 

o 

Li. 

CC 

UJ 

GL 


—  ep 

o  c 

CO  ft) 

•  C/J  w 

£  *T3 

SSi? 

«-■  9  <8 
.  (3  JS 

ill 

«  aA 
aUT> 

o 


c  £ 

<u  “ 


*5  CO 
W  a!  ^ 

•H 


9 

e 

«rt  4J  X 

83  «  “ 
e  a  .s  & 
,2o  85 

5  w  P  ^ 

•**  “  u 

a»  a>  ►>  2 

V  U  W  2 

■"  e  J  5 

~  ~  S 

P  C  (A  C 

r  P  (A  o 

lii^u 

►.u  £'S  I 
•g'S  P52 
Sp-<  gc 

•-•as" 

affa&K 

Sls.il 

5'gS‘Sl 

•3  B  «9 

2  m  rt  v  Of  g 

.5  a!  IS  -0  63  fc 

&£  3  v  o'S  5 
«  T3  g  C  .9  jQ 

S  -3  g  Z  2  a 

IS  w  E  o  —  ^ 


W  ’ -  vw 

.2  a  c  -o  s 
a  8  s  &•§ 

Jiffs  6*8 

5  *3  *  9  O 


2 

5 

O. 

V 

o 


co  •-*  « 

•s  V  «C 
<2  «-•  ** 
g  9  © 

O  3  *S 
v  S» 

'I  5r  co 

2-S  c  , 
£*•23 

2  §32 

•2  gp*C  ? 

2 .S3  2 
o  o  5  © 
ws  SS 


•^.5  X  „ 

~*§  "  O  W  CO 

c  2  «  «-  « 

EES  g.22 

‘r  u9  g  «*3 
5  9  c-o  &-2 

8.»>"s  2  8 
"  &  a  S  § 

mS  4)  ►,«  w 

•a  S*  S-0  E 


.5  M=c«-  i 

IS  eij:.<2  S 
2.5  a  93  a  o 
©  o  a  «  a  *- 


OO  ©  o  B  = 
W  T3  W  •>»  «  & 


r 

. 


53 


INITIAL  STANDARD  SETTING 
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Assessment  of  acceptability  of  effective  and  ineffective 
behaviors  described  in  critical  incident  workshops 


Soldier  Performance  Distribution  by  MOS, 
Performance  Factor,  and  Judgment  Method 
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71L  Filing  Direct  47  10.3  13.0  10.8  14.4 

Task  50  35.7  18.7  8.0  7.9 

Ratings  52  4.6  12.4  4.8  5.6 
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COMPUTERIZED  ADAPTIVE  SCREENING 

TEST 
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ARMY  RESEARCH  INSTITUTE 
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IMPLEMENTING  SPECIAL  SCREENING  TESTS 


CLINTON  WALKER 
ARMY  RESEARCH  INSTITUTE 
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S3  (SKILLS  SELECTION  AND  SUSTAINMENT)  PROGRAM 
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Project  A  Improvements  to  TOW  Gunnery 
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Percent  of  11H  Requiring  More  Than 
3  Tables  to  Qualify  by  PScomp  Cut  =  60 
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Number  of  Trials 


S3:  TOW  GUNNERY  AT  BENNING 
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11H  TOW  GUNNERY:  LIVE-FIRE  EXERCISE 
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S3  -  ALL  ARMOR  MOSs 
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Spatial  Track  Spatial/Psychomotor  ABLE 

Predictor  Tests 


FORT  KNOX:  TANK  GUNNERY 

Correlations  With  Speed/ Accuracy  Outcome 
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Note:  N  •  496 


S3  ICOFT  PERFORMANCE 


Low  1/3  Mid  1/3  High  1/3 

Performance  on  S3  Psychomotor  Scale 


STEPWISE  REGRESSIONS  ON  SPEED/ACCURACY  OUTCOME 
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GT  .34 

Spatial/psychomotor  .55 
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S3  PILOT  AT  FORT  BLISS 

Outcomes  to  date 
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S3  AT  KNOX:  OUTCOME  OF  PILOT 


IMPLEMENTATION  IN  USAREUR 


S3/USAREUR:  NEXT  STEPS 


Work  for  implementation  of  spatial/psychomotor 
tests  in  the  MEPS 


IMPLEMENTATION  AT  FORT  SILL 


NEXT  IMPLEMENTATION 

SELECTION  INTO  TRAINING  FOR  SPECIAL  FORCES 
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THE  AIR  FORCE  PROGRAM 


MAJOR  FRANK  T.  VACARRO,  USAF 
HQS  AIR  FORCE  SYSTEMS  COMMAND 


87 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AIR  FORCE  SYSTEMS  COMMAND 
ANDREWS  AIR  FORCE  BASE  DC  20334-5000  ' 


REP'_>  TO 

AT-Nor  x?H  IQ  Deo  88 


Information  for  Dr  Alluisi  on  funding  in  PF  0607205F 


8A.P/AQT  (Lt  Col  Higgins) 

I.  As  of  Q  ‘Joy  88,  AFHPL  was  scheduled  to  receive  the  following  funds  in 
Program  dement  06022Q5F  for  wv  8Q  thru  FY  94 : 


FY  1989  FY  1990  FY  1991  FY  1992  FY  199?  FY  1994 

a.  HRTj/Qrn  Flving  Training  Qimulators,  Part-Task  dainers  (strong  ^AC  support). 
Proiects  IIP? ,  119°,  6114  Q464  8594  8557  QQ58  9758  9855 

h.  HP.L/LR  Logistics  and  Command  \  Control  Training  (strong  AdC  an^  PACAF 
support ) . 

Projects  1710  &  5017  50?5  451^  4484  495^  4975  5050 

c.  HRL^TD  Computer-based  Training,  Tr!1T_nin<j  Decision  Aids,  dsic  Job  Skills 
'strong  ATC  support,  reasonably  strong  TAC  support). 

Projects  11°1  h  nrr*>L  PQ50  2789  7778  5745  d?5  5200 

d.  FRL/MO  Selection,  Classification,  Assignment,  Pianpower  #  ^o^ce  Models,  Job 
Performance  Measures,  MPm  Integration  (lM°A0m8)  Tools  (strong  ATC  H.  AW/DP 

juppo  rt ) . 

Project  77iq  ^10  5054  5051  7597  5520  5400 

2.  While  actual  program  reductions/cancellations  as  a  result  of  the  4)1.8**  PBD 
action  have  not  vet  been  finalised,  the  basic  approach  will  be  to  target  those 
“ffo-M  most  close1. v  related  to  the  4SVAB  B&D.  Thus,  the  bulk  of  the  reductions 
will  be  in  the  MO  division.  It  now  appears  that  the  six  manpower  slots  will 
also  he  taken  from  pF  0507o95d 

a.  MO  is  responsible  for  the  following  programs  which  will  be  impacted  by  ^BD 
055C.  Borne  efforts,  such  as  the  aircrew  selection  5  classification  and  the  **PT 
Integration  tools,  are  responding  to  major  Air  Force  requirements  and  must 
continue  to  be  funded .  {*  Indicates  efforts  likelv  to  be  cancelled.) 


*A.5VAn 

400 

455 

500 

200 

700 

700 

*AF9QT  /other 

71 

100 

100 

100 

50 

50 

^Learning  Anilities  Meas. 

575 

541 

6 

570 

284 

780 

UNITED  STATES  AIR  FORCE 


SEPTEMBER  18.1947 
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Basic  Job  Skills 

732 

144 

Person-Job  Match 

100 

225 

200 

Occupational  Measurement 

200 

250 

750 

250 

Aircrew  Selection  h  Classif 

614 

464 

4^5 

4^5 

405 

500 

MPT  Integration  Tools 

463 

600 

1255 

1000 

740 

220 

b.  Additionally,  the  Job  P 

'erformance 

Measurement  effort  managed 

bv  the 

ID 

division,  which  is  determining  the  links  between 

ASVAB 

and  job  performance 

will  likelv  be  signif icantlv 

reduced  or 

cancelled . 

Job  Performance  Measures 

585 

500 

600 

855 

750 

600 

3.  The  ^YOO/Ol  Descrioti ve  Nummary  does  not  reflect  the  likelv  distribution 
of  the  PBD  reduction,  but  rather  spreads  the  reduction  across  several 
projects.  We  expect,  based  upon  the  comments  of  Dr  Alluisi  and  Dr  Selman, 
that  some  of  the  funds  taken  incorrectly  by  PBD  033C  will  be  restored.  A 
large  reduction  in  nroject  7719  now  followed  by  a  plus  up  next  cycle  could 
draw  extra  attention  from  the  Congressional  staffers  and  give  the  impression 
that  we  are  starting  a  major  new  effort.  We  are  also  embarking  on  a  major 
change  in  the  process  that  we  (AFSC)  use  to  determine  the  investment  strategy 
for  MPT  efforts.  This  new  process  will  focus  our  investment  on  those 
technology  needs/requirements  which  the  "users"  indicate  are  the  most 
important  to  them. 


, ,  *  ... . 

TRANK  T.  VACCARO,  Major,  USAF 
Chief,  Personnel,  Training  5  Simulation 
Directorate  of  Combat  Support 
DCS/Technology  3  Requirements  Planning 


ENLISTED  SELECTION  AND  CLASSIFICATION 
TESTING  RESEARCH  AND  DEVELOPMENT 


MALCOLM  REE 

AIR  FORCE  HUMAN  RESOURCES  LABORATORY 


ENLISTED  SELECTION  &  CLASSIFICATION  TESTING 
RESEARCH  AND  DEVELOPMENT 
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BRIEFER:  DR  REE 

DECEMBER  1988 


Enlisted  Selection  and  Classification 
Testing  Research  and  Development 
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ASVAB  Background 
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ASVAB  Programs 
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Joint  Service  ASVAB  5- Year  R&D  Plan 


Joint  Service  ASVAB  5- Year  R&D  Plan 
Test  Development/Test  Equity 


OINT  SERVICE  ASVAB  5-YEAR  R&D  PLAN 
TEST  DEVELOPMENT/TEST  EQUITY 
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JOINT  SERVICE  ASVAB  5- YEAR  R&D  PLAN 
TEST  DEVELOPMENT/TEST  EQUITY 
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Joint  Services  ASVAB  5- Year  R&D  Plan 
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Joint  Services  ASVAB  5-Year  R&D  Plan 
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Joint  Service  ASVAB  5- Year  R&O  Plan 
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Joint  Services  ASVAB  5- Year  R&D  Plan 
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Studies  in  Planning  Phase 


Summary 


LEARNING  ABILITIES  MEASUREMENT 

PROGRAM 


PATRICK  KYLLONEN 

AIR  FORCE  HUMAN  RESOURCES  LABORATORY 
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Present  R&D  Program  Status 

Background 
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—Aptitude  testing  not  driven  by  current  models 
-Technology  Gap 
Gap  between... 

—What  jobs  require  (high  technology) 

—What  tests  measure  (low  technology) 


Present  R&D  Program  Status 

Background 


Present  R&D  Program  Status 

Approach 
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>mpare  LAMP  ability  factors’  predictive  value  against 
-ASVAB,  BAT.  AFOQT 


Present  R&D  Program  Status 

Four-Sources  Framework 
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Present  R&D  Program  Status 

Conclusions  from  a  Validity  Study 
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Accomplishments 

Facilities 


£8 

o  CO 

©*> 

If 

(0  X) 
©  —  . 

©5® 

©  5  4: 

■H  <  < 
T3  w 


± -o 3 w  2  2*^0  o 

;=  3  £  ©  ®  SS  *J  *3 


8  £  s  § 

ll  g  £  ■- 

-  «  <D  CO 


*5  co 

15 

•So 

3 


.2  »  §  « 

O  C  «S 

*  ?2f 

*  -s  •5  2 
5  ?  =  a 

O  ©  jO  C 

T5  <  O 

©  .  a 

3  S.|® 

*-  *£1  CO 

•—  n 
a  c  c  £ 

<5  o  ° 

o  ©o® 
©  co  ..  © 

.©  «j  E  i 
3  o  ©  o 
©0  ZO 

©  «  *  » 
UJ 


©  8 
S  u- 

‘C  c 
©  o 

l  I 

E  2 

8  3 

i  2 
3  2 


5  e 

£  I 

2  © 

2  H 

a  ■© 
■O  © 

©  JC 

a  © 

I  5 

I  2 

©  © 

a  uj 

•  • 


'A  c 

a  © 
©  E 

>  © 
5  © 

2$ 

?< 

•5  g 
Sc 

2| 

2  X 
T3  2 

©  a 
£  £ 
«  © 
i©  O 

*  i  • 

••  © 

©  E 
o  © 

0  Z 


©  o 

f  * 

a  c» 
£2 
O  A 
O  . 
^  GQ 


i  i  i 


122 


Developed  prototype  CLASS  courses  for  validation  studies 


Accomplishments 
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Accomplishments 

Publications 
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Future  Plans 

Basic  Research 
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Processing  capacity  vs. 

probability  of  creating/strengthening  a  memory  trace 


LEARNING  ABILITIES  TAXONOMY 
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RESULTING  KNOWLEDGE  TYPE 


Future  Plans 

Technology  Transition  Planning 
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FUNDING  FOR  NAVY  PROGRAM 


JOHN  J.  PASS 

NAVY  PERSONNEL  RESEARCH  AND  DEVELOPMENT 

CENTER 


129 


FUNDING 
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ACCELERATED  CAT-ASVAB  PROJECT 


WILLIAM  A.  SANDS 

NAVY  PERSONNEL  RESEARCH  AND  DEVELOPMENT 

CENTER 
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ACCELERATED  CAT-ASVAB  PROJECT 


-9  December  1988 


OVERVIEW 
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Conclusion 


CAT-ASVAB  Program 
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I 


•  Evaluate  CAT-ASVAB  as  a  potential  replacement  for  the 
paper-and-pencil  version  of  the  battery  (P&P-ASVAB) 


Armed  Services  Vocational  Aptitude  Battery 

(ASVAB) 
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Computerized  Testing  Systems 
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Illustrative  Five  Item  CAT  Test 


Incorrect 


Utilization-of  Test  Items 


CAT  HISTORY 
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U.S.  Civil  Service  Commission  (USCSC)  Development 
of  a  CAT  Version  of  the  Professional  and 
Administrative  Career  Examination  (PACE) 

Joint  Sponsorship  of  First  Major  CAT  Conference 
by  ONR  and  USCSC 


CAT  HISTORY 
(continued) 
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CAT  HISTORY 
(continued) 
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ECAT  POA&M  Written 

Emphasis  on  New  Predictors  to  Augment  ASVAB  Tests 
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PREVIOUS  APPROACH 
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Stage  3  *  Full  Scale  Production,  Deployment, 
&  Implementation 
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Score  Equating  Verification 


Purpose:  Evaluate  Human-Computer  System  Interaction 
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Other:  Instructions  Revised 
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Medium  of  Administration 
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Group  1 --Obtain  Computer-Based  Calibration 
Group  2-Obtain  Paper-and-Pencil  Based  Calibration 
Group  3--Each  Calibration  Used  to  Estimate  Ability 


Medium  of  Administration 
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Comprehension  Under  Consideration 


Cross-Correlation 
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Group  1— CAT-ASVAB  Form  1,  Then  CAT-ASVAB  Form  2 
Group  2-P&P-ASVAB  Form  9B,  Then  P&P-ASVAB  Form  10B 
Second  Test:  About  5  Weeks  After  First  Test 


Cross-Correlation 
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Data  Base  Development 

•  Under  Construction 

•  Analyses— Scheduled  Start  Nov  1988 


Preliminary  Operational  Check 
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Preliminary  Operational  Check 
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Merge  and  Edit  ACAP  Results  on  System-80 
Telecommunicate  ACAP  Results  to  USMEPCOM  Headquarters 


Score  Equating  Development 
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Group  2--CAT-ASVAB  Form  2,  Then  Operational  P&P-ASVAB 
Group  3--P&P-ASVAB  Form  8A#  Then  Operational  P&P-ASVAB 


SCORE  EQUATING  DEVELOPMENT 
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•  Microcomputer  Performance  —  Satisfactory 

•  Logistics  —  No  Problems 


Score  Equating  Verification 
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Equipercentile  Equating  for  Subsequent  Operational  Use 
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Seattle  01  Aug  90  31  Oct  90 

Boston  01  Oct  90  31  Dec  90 

Omaha  03  pec  90  28  Feb  91 

Jackson  01  Feb  91  30  Apr  91 


CAT  -  ASVAB  Advantages 
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Future  Tests 


CAT  -  ASVAB  Advantages 
Administrative 
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Simplified  Revision 


CAT  -  ASVAB  Advantages 
Scoring 
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CAT  -  ASVAB  Advantages 
Precision 
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Same  at  mid-range  ability  levels 


CAT  -  ASVAB  Advantages 
Security 
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Items  stored  in  volatile  memory 


CAT  -  ASVAB  Advantages 
Future-Tests 
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Response  latency  measurement 


CAT-ASVAB  Ongoing  Program 
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ACCELERATED  CAT-ASVAB  PROGRAM 
PSYCHOMETRIC  ACCOMPLISHMENTS 


BRUCE  BLOXOM 

DEFENSE  MANPOWER  DATA  CENTER 


DEPARTMENT  OF  DEFENSE 

MANPOWER  DATA  CENTER 


REPLY  TO  DMDC 

C  1 600  N  WILSON  BLVO  SUITE  400 
ARLINGTON  VIRGINIA  22209  2593 
>C  550  C  A  Ml  NO  EL  ESTERO  SUITE  200 
MONTEREY  CALIFORNIA  93940  3231 
C  210OGARDEN  ROAD  SUITE  J 

MONTEREY  CALIFORNIA  93940-5316 


12  December  1988 


MEMORANDUM  FOR  DR.  EARL  A.  ALLUISI 

SUBJECT:  Minutes  of  Selection  and  Classification 

Topical  Area  Review 

ENCLOSURE:  Briefing  Slides,  Technical  Addendum  and  References 

Enclosed  are  clean  copies  of  the  briefing  slides  and  technical 
addendum,  plus  the  references  which  you  requested,  for  my 
presentation  at  the  Selection  and  Classification  Topical  Area 
Review  which  was  held  on  8-9  December  1988  at  IDA.  They  are  for 
inclusion  in  the  minutes  of  the  Review. 

Thank  you  for  the  opportunity  to  present  the  work  of  the  CAT- 

ASVAB  technical  Committee. 


Bruce  Bloxom 

Personnel  Security  Data  and 
Special  Studies  Branch 
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Accelerated  CAT— ASVAB  Program 
Psychometric  Accomplishments 


Dr.  Bruce  Bloxom 
Chairman,  Technical  Committee 
CAT— ASVAB  Working  Group 

Briefing  Presented  To 

Assistant  for  Training  and  Personnel  Systems  Technology 

(ODDDR&E/R&AT) 

and 

Director  for  Accession  Policy 
(OASD/FM&P) 

9  December  1983 
Washington,  DC 
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Overview  of  Briefing 


Background 

General  Purpose  of  ACAP  Psychometric  Studies 
Types  of  ACAP  Psychometric  Studies 


Evaluation  of  Accomplishments  of  Studies 
Method  of  Evaluation 
Results  of  Evaluation 

Summary  of  Research  Bases 
Summary  of  Output  of  Studies 
Compilations  of  Studies 
Conclusions 
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Background 


GENERAL  PURPOSE  OF 
ACAP  PSYCHOMETRIC  STUDIES 


Develop  additional  psychometric  knowledge  which  is 
necessary  for  Full-Scale  Development  (FSD)  of 
Computerized  Adaptive  Testing  Version  of  the 
Armed  Services  Vocational  Aptitude  Battery 
(CAT-ASVAB) 
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Background 


TYPES  OF  ACAP  PSYCHOMETIC  STUDIES 


Procedures  for  Development  of  CAT-ASVAB  Forms 
Evaluate  Influences  on  CAT-ASVAB  Score  Precision 
Evaluate  Equating  of  CAT-ASVAB  and  ASVAB 
Evaluate  Validity  of  CAT-ASVAB 


182 


Evaluation  of  Accomplishments 


METHOD  OF  EVALUATION 


1.  Compile  information  provided  to 

CAT— ASVAB  Technical  Committee 

2.  Classify  studies  reported  or  briefed 

since  start  of  ACAP 

3.  Identify  research  basis  of  each  study 

4.  Identify  output  of  each  study 


i 
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Evaluation  of  Accomplishments 

METHOD  OF  EVALUATION 
Classification  of  Studies 

Procedures  for  Development  of  Forms  (25  Studies) 

-  Item  Pools 

-  Adaptive  Testing  Procedures 

-  Speeded  Test  Procedures 

-  Pre-Test  of  CAT-ASVAB 

-  Equating  CAT-ASVAB  and  ASVAB 

Evaluate  Influences  on  Score  Precision  (6  Studies) 

-  Influences  on  Item  Calibration 

-  Comparison  of  CAT-ASVAB  and  ASVAB 
Evaluate  Equating  of  CAT-ASVAB  and  ASVAB  (4  Studies) 

-  Equivalence  of  Results  by  Form, 

Subgroup  and  Condition 

Evaluate  Validity  of  CAT-ASVAB  (3  Studies) 

-  Correlations  Among  Subtests 

-  Predictive  Validity 
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Evaluation  of  Accomplishments 


METHOD  OF  EVALUATION 


Types  of  Output  of  Studies 

Use  in  CAT-ASVAB  for  ACAP 
Report  or  Publication 
Briefing  of  Results 

-  CAT-ASVAB  Technical  Committee  (CTC) 

-  Defense  Advisory  Committee  (DAC) 

-  Conference  Paper  (CONF) 

Collection  of  Data 

Briefing  of  Analysis  Plans 

-  CAT-ASVAB  Technical  Committee  (CTC) 

-  Defense  Advisory  Committee  (DAC) 

Contract  Deliverable 
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Evaluation  of  Accomplishments 

< 


RESULTS  OF  EVALUATION 


Summary  of  Research  Bases 


Type  of  Study 

Type 

of  Research  Basi 

ONR 

Other 

Stand. 

Procedures  for 
Development  of  Forms 

Work 

Unit 

Put. 

cation 

Method 

Item  Pools 

5 

0 

4 

Adaptive  Testing 

4 

3 

1 

Speeded  Testing 

1 

0 

0 

Pre-Test 

2 

0 

0 

Equating 

3 

1 

1 

Evaluate  Precision 

6 

0 

0 

Evaluate  Equating 

0 

0 

4 

Evaluate  Validity 

0 

1 

2 

Total 

21 

5 

12 

Total  Studies  38 


Evaluation  of  Accomplishments 


RESULTS  OF  EVALUATION 

Summary  of  Output  of  Studies 

Type  of  Study  Type  of  Output 

Use  Report  Brief  Collect  Brief  Contract 


in 

Procedures  for 
Development  of 
CAT-ASVAB  Forms 

CAT 

Results 

Data 

Plans 

Deliver. 

-  Item  Pools 

8 

2 

5 

— 

— 

1 

-  Adaptive  Testing 

8 

0 

8 

— 

— 

0 

-  Speeded  Testing 

1 

0 

1 

— 

— 

0 

-  Pre-Test 

1 

1 

2 

— 

— 

1 

-  Equating 

3 

0 

5 

— 

— 

0 

Evaluate  Precision 

— 

1 

2 

3 

2 

0 

Evaluate  Equating 

— 

0 

0 

3 

4 

0 

Evaluate  Validity 

— 

0 

2 

0 

1 

2 

Total  21  4  25  6  7  4 

Total  Studies  38 
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Evaluation  of  Accomplishments 


RESULTS  OF  EVALUATION 

Compilations  of  Studies 
Activity  Document 

Development  of  Forms  of  Psychometric 

CAT-ASVAB  for  ACAP  Decision  List 

Five  Data  Collections 

-  Joint-Service  Contract 

Validity  Study  Deliverable 

-  Score  Equating  Development  Data  Analysis 

Plan 

-  Cross-Correlational  Check  Data  Analysis 

Plan 

-  Medium  of  Administration  Data  Analysis 

Plan 

-  Score  Equating  Verification  Data  Analysis 

Plan 
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Evaluation  of  Accomplishments 


CONCLUSIONS 


1.  ACAP  psychometric  studies  are  based  on 
research  studies  and  standard  methods. 

2.  Studies  of  procedures  for  development  of 
CAT-ASVAB  will  result  in  psychometrically 
sound  CAT-ASVAB  forms. 

3.  Joint-Service  Validity  Study  and  studies 
now  in  progress  will  address  questions  of 
psychometric  feasibility  of  CAT-ASVAB  as  a 
substitute  for  ASVAB  in  the  near  term  and 
in  Full-Scale  Development. 
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PSYCHOMETRIC  ACCOMPLISHMENTS 


OF  ACCELERATED  CAT-ASVAB  PROGRAM  (ACAP) 


> 


► 

TECHNICAL  ADDENDUM  TO  BRIEFING 


► 


► 


► 


► 


► 
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PROCEDURES  FOR  DEVELOPMENT  OF  CAT-ASVAB  FORMS 


ITEM  POOLS 


INITIAL  DEVELOPMENT 

STUDY:  DIMENSIONALITY  OF  ASVAB  SUBTESTS 

BASIS:  FULL  INFORMATION  FACTOR  ANALYSIS  (NR  475-018) 

OUTPUT:  USE  IN  CAT-ASVAB 

STUDY:  PRESTWOOD,  VALE,  MASSEY  AND  WELSH  ITEM  POOL 
BASIS:  EXISTING  ASVAB  ITEM  TYPES  (STANDARD  METHOD) 

OUTPUT:  USE  IN  CAT-ASVAB,  REPORT  ( AFHRL  TR-85-19) 

STUDY:  EXPERIMENTAL  CAT-ASVAB  ITEM  POOLS 

BASIS:  EXISTING  ASVAB  ITEM  TYPES  (STANDARD  METHOD) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

ITEM  CALIBRATION 

STUDY:  PRESTWOOD,  VALE,  MASSEY  AND  WELSH  CALIBRATION 
BASIS:  JOINT  MAXIMUM  LIKELIHOOD  ( ONR  67/18,  LORD) 

OUTPUT:  USE  IN  CAT-ASVAB,  REPORT  (AFHRL  TR-85-19) 

STUDY:  CALIBRATION  OF  EXPERIMENTAL  CAT-ASVAB  ITEM  POOLS 
BASIS:  JOINT  MAXIMUM  LIKELIHOOD  (ONR  67/18,  LORD) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

ITEM  REVIEW 

STUDY:  ETS  REVIEWS  OF  CONTENT,  QUALITY  AND  SENSITIVITY 
BASIS:  SUBJECTIVE  JUDGEMENTS  (STANDARD  METHOD) 

OUTPUT:  CONTRACT  DELIVERABLE 

STUDY:  NPRDC  REVIEWS  OF  CONTENT,  QUALITY  AND  SENSITIVITY 
BASIS:  ETS  REVIEWS  PLUS  ITEM  STATISTICS  (STANDARD  METHOD) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

ASSIGNMENT  TO  FORMS 

STUDY:  OPTIMIZE  SIMILARITY  OF  POOL  INFORMATION  FUNCI  i.ONS 
BASIS:  TEST  INFORMATION  (NR  042-249,  BIRNBAUM) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

FINAL  ITEM  POOLS 

STUDY:  RETAIN  ITEMS  USED  IN  SIMULATED  ADAPTIVE  TESTING 
BASIS:  ADAPTIVE  ABILITY  TESTING  (NR  150-431,  WEISS) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 


i 
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PROCEDURES  FOR  DEVELOPMENT  OF  CAT-ASVAB  FORMS 


ADAPTIVE  TESTING  PROCEDURES 


ITEM  SELECTION 

SELECTION  CRITERION 

STUDY:  METHOD  FOR  CHOOSING  NEXT  ITEM 

BASIS:  ITEM  INFORMATION  FUNCTION  (SAMEJIMA,  1969) 

OTITPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

EXPOSURE  CONTROL 

STUDY:  LIMIT  ITEM  EXPOSURE  IN  CAT-ASVAB 

BASIS:  ADAPTIVE  ABILITY  TESTING  (NR  150-431,  WEISS) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

CONTENT  BALANCING 

STUDY:  ITEM  POOL  FACTOR  ANALYSES 

BASIS:  FULL  INFORMATION  FACTOR  ANALYSIS  (NR  150-541) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 


SUBTEST  TIME  LIMITS 

STUDY:  ESTIMATE  TESTING  TIME  OF  SLOW  EXAMINEES 
BASIS:  FIT  OF  LOGNORMAL  DISTRIBUTION  (STANDARD  METHOD) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

TEST  SCORING 

INTERIM  SCORE 

STUDY:  METHOD  OF  SCORING  BETWEEN  ITEMS 

BASIS:  APPROXIMATION  TO  POSTERIOR  MEAN  (OWEN,  1976) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

FINAL  SCORE 

STUDY:  METHOD  OF  SCORING  AT  END  OF  TEST 
BASIS:  POSTERIOR  MODE  (SAMEJIMA,  1969) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 

PENALTY  FOR  INCOMPLETE  TEST 

STUDY:  SCORE  REDUCTION  FOR  VERY  SLOW  EXAMINEES 
BASIS:  ADAPTIVE  ABILITY  TESTING  (NR  150-431) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC 

COMBINING  AI  AND  SI  SUBTESTS 

STUDY:  ONE  SCORE  TO  EQUATE  WITH  ASVAB  AS  SUBTEST 
BASIS:  TEST  CHARACTERISTIC  CURVE  (NR  042-249,  BIRNBAUM) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC 
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PROCEDURES  FOR  DEVELOPMENT  OF  CAT-ASVAB  FORMS 


SPEEDED  TEST  PROCEDURES 


TEST  SCORING 

STUDY:  METHOD  OF  SCORING  FROM  ITEM  RESPONSE  TIMES 
BASIS:  STUDIES  OF  SCORING  METHODS  (NR  4421-534,  GREEN) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 
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PROCEDURES  FOR  DEVELOPMENT  OF  CAT- ASVAB  FORMS 


PRE-TEST  OF  CAT- ASVAB 

SUBJECTIVE  RESPONSES  OF  EXAMINEES 

STUDY:  EXPERIMENTAL  CAT-ASVAB  ADMINISTERED  TO  RECRUITS 
BASIS:  EFFECTS  ON  MOTIVATION  (NR  150-382,  WEISS) 

OUTPUT:  DELIVERABLE,  BRIEF  CTC  AND  DAC , 

REPORT  ( CM A  RM-86-151) 

STUDY:  ACAP  CAT-ASVAB  WITH  RECRUITS  &  HIGH  SCHOOL  STUDENTS 
BASIS:  EFFECTS  ON  MOTIVATION  (NR  150-382,  WEISS) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC,  DAC  AND  CONFERENCE 
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PROCEDURES  FOR  DEVELOPMENT  OF  CAT-ASVAB  FORMS 


EQUATING  CAT-ASVAB  AND  P&P-ASVAB 


DATA  EDITING 

DIFFERENCE  SCORES 

STUDY:  SELECT  UNMOTIVATED  EXAMINEES 

BASIS:  CLASSICAL  TEST  THEORY  (NR  151-201,  LORD/NOVICK) 
OUTPUT:  BRIEF  CTC 

APPROPRIATENESS  MEASUREMENT 

STUDY:  SELECT  EXAMINEES  WITH  ABERRANT  DATA 

BASIS:  INDIVIDUAL  MODELLING  (NR  154-445,  LEVINE  ET  AL ) 

OUTPUT:  BRIEF  CTC 

DISTRIBUTION  SMOOTHING 

STUDY:  SMOOTHING  CAT-ASVAB  AND  P&P-ASVAB  DISTRIBUTIONS 
BASIS:  WORK  AT  NPRDC ,  PLUS  KRONMAL  AND  TARTER  (1968) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC 

SUBTEST  EQUATING 

STUDY;  EQUATING  CAT-ASVAB  AND  ASVAB  SUBTESTS 
BASIS:  EQUIPERCENTILE  (NR  150-463,  GREEN  ET  AL ) 

OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC  AND  DAC 


COMPOSITE  EQUATING 

STUDY:  EQUATING  CAT-ASVAB  AND  ASVAB  SUBTESTS 

BASIS:  IDENTITY,  LINEAR  AND  EQUIPERCENTILE  (STANDARD  METHOD) 
OUTPUT:  USE  IN  CAT-ASVAB,  BRIEF  CTC 
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EVALUATE  INFLUENCES  ON  SCORE  PRECISION 


EFFECT  OF  CALIBRATION  MEDIUM  OF  ADMINISTRATION 

STUDY:  EFFECT  OF  PARAMETER  SPECIFICATION  ERRORS 
BASIS:  ROBUSTNESS  OF  ADAPTIVE  TEST  (NR  150-433,  WEISS) 
OUTPUT:  BRIEF  CTC  AND  DAC 

STUDY:  ANALYSIS  OF  EXPERIMENTAL  CAT-ASVAB  DATA 
BASIS:  ITEM  RESPONSE  THEORY  (NR  151-201,  LORD/NOVICK) 
OUTPUT:  BRIEF  CTC  AND  DAC,  REPORT  ( CNA  RM-86-189) 

STUDY:  MEDIUM- OF- ADMIN I STRATI ON  STUDY  AT  NAVY  RTC 
BASIS:  MAXIMUM  LIKELIHOOD  ESTIMATION  ( ONR  67/18,  LORD) 
OUTPUT:  DATA  COLLECTED,  PLAN  BRIEFED  TO  CTC  AND  DAC 

EFFECT  OF  USING  ADAPTIVE  DATA  FOR  CALIBRATION 
STUDY:  PRECISION  OF  ONLINE  CALIBRATION 
BASIS:  MARGINAL  MAXIMUM  LIKELIHOOD  (NR  875-018,  BOCK) 
OUTPUT:  DATA  COLLECTED  (SCORE  EQUATING  DEVELOPMENT) 

EFFECT  OF  OPERATIONAL  MOTIVATION  ON  CALIBRATION 

STUDY:  COMPARISON  OF  ACAP  ON-LINE  CALIBRATIONS 
BASIS:  MARGINAL  MAXIMUM  LIKELIHOOD  (NR  875-018,  BOCK) 
OUTPUT:  HALF  OF  DATA  COLLECTED 

EFFECT  OF  CAT  VERSUS  CONVENTIONAL  TESTING 

STUDY:  ALTERNATE  FORM  RELIABILITY  OF  CAT-  AND  P&P-ASVAB 
BASIS:  TEST  THEORY  (NR  151-201,  LORD  AND  NOVICK) 

OUTPUT:  DATA  COLLECTED,  PLAN  BRIEFED  TO  CTC  AND  DAC 
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EVALUATE  EQUATING  OF  CAT-ASVAB  AND  P&P-ASVAB 


EQUIVALENCE  OF  RESULTS  BY  CAT-ASVAB  FORM 

STUDY:  COMPARE  EQUATING  RESULTS  OF  TWO  CAT-ASVAB  FORMS 
BASIS:  EQUIPERCENTILE  EQUATING  ( STANDARD  METHOD ) 

OUTPUT:  DATA  COLLECTED,  PLAN  BRIEFED  TO  CTC  AND  DAC 

EQUIVALENCE  OF  RESULTS  BY  SUBGROUP 

STUDY:  COMPARE  RESULTS  FOR  WOMEN  AND  FOR  BLACKS 
BASIS:  EQUIPERCENTILE  EQUATING  (STANDARD  METHOD) 

OUTPUT:  DATA  COLLECTED,  PLAN  BRIEFED  TO  CTC  AND  DAC 

EQUIVALENCE  OF  RESULTS  BY  MOTIVATIONAL  CONDITION 

STUDY:  COMPARE  OPERATIONAL  AND  NON- OPERATIONAL  RESULTS 
BASIS:  EQUIPERCENTILE  EQUATING  (STANDARD  METHOD) 

OUTPUT:  HALF  OF  DATA  COLLECTED,  PLAN  BRIEFED  TO  CTC  AND  DAC 

EQUIVALENCE  OF  RESULTS  BY  ASVAB  TESTING  EXPERIENCE 

STUDY:  COMPARE  RESULTS  FOR  RETESTS  AND  INITIAL  TESTS 
BASIS:  EQUIPERCENTILE  EQUATING  (STANDARD  METHOD) 

OUTPUT:  DATA  COLLECTED,  PLAN  BRIEFED  TO  CTC 


EVALUATE  VALIDITY  OF  CAT- AS VAB 


CONVERGENT  AND  DISCRIMINANT  VALIDITY 

STUDY:  CAT- ASVAB  AND  ASVAB  SUBTEST  CORRELATIONS 
BASIS:  FACTOR  ANALYSIS  (STANDARD  METHOD) 

OUTPUT:  DELIVERABLE,  BRIEF  CTC ,  DAC  AND  CONFERENCE 

STUDY:  CAT-ASVAB  AND  P&P-ASVAB  COVARIANCE  STRUCTURES 
BASIS:  STRUCTURAL  EQUATION  MODELLING  (JORESKOG,  1973) 
OUTPUT:  PLAN  BRIEFED  TO  CTC  AND  DAC 


PREDICTIVE  VALIDITY 

STUDY:  CAT-ASVAB  AND  P&P-ASVAB  PREDICTION  OF  TRAINING 
BASIS:  MULTIPLE  LINEAR  REGRESSION  (STANDARD  METHOD) 
OUTPUT:  DELIVERABLE,  BRIEF  CTC,  DAC  AND  CONFERENCE 
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UTILITY  ANALYSIS 
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OVERSIGHT  ORGANIZATION 
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PSYCHOMOTOR  ABILITIES  REVIEWED 

PRELIMINARY  VALIDATION  OF  NEW  REASONING  TESTS 
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aDOES  NOT  INCLUDE  DOD-WIDE  VALIDATION  PROGRAM 
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OVERLAPPING  PROCESSES 

WORKING  MEMORY  OR  ATTENTIONAL  SOURCE 
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FUTURE  TESTS  -  DEVELOPMENT  FLOW 


REDIRECTION  OF  COMPUTERIZED  ADAPTIVE 

TESTING 
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PARALLEL  DEVELOPMENT  OF  ACAP  AND  ECAT 
(October  18,  198S) 
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IDENTIFY  SIX  TO  TEN  MILITARY  JOBS  FOR  TEST  VALIDATION 

ASSIST  IN  COORDINATION  OF  VALIDATION  DATA  COLLECTION 
AT  RECRUIT  TRAINING  CENTERS 


COMPUTERIZED  TESTING  PROGRAMS? 


CAT-ASVAB  PROGRAM 
CONCEPT  OF  OPERATION  AND  COST 
COST/BENEFIT  ANALYSES 


CLESSEN  J.  MARTIN 

OFFICE  OF  THE  CHIEF  OF  NAVAL  OPERATIONS 
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CAT-ASVAB  PROGRAM  CONCEPT  OF  OPERATION 
AND  COST/BENEFIT  ANALYSIS 


Other  Considerations 
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CAT-ASVAB  PROGRAM  CONCEPT  OF  OPERATION 
AND  COST/BENEFIT  ANALYSIS 
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Identify  and  Characterize  Operational  Problems 
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P&P  CENT  HVS  CASCAT  MOBILE 

Alternative 


CAT-ASVAB  ECONOMIC  BENEFITS  ANALYSIS 
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LIFE  CYCLE  COST  ESTIMATES  FOR  P&P  ASVAB  AND  CAT-ASVAB 
ALTERNATIVES:  DRAWN  FROM  CAT-ASVAB  ECONOMIC  ANALYSIS 


CAT-ASVAB  PROGRAM  CONCEPT  OF  OPERATION 
AND  COST/BENEFIT  ANALYSIS 


Computerized  Adaptive  Screening  and  CAT-ASVAB  Testing 


CAT-ASVAB  ECONOMIC  ANALYSIS 
OPERATIONS  CONCEPT  LIFE  CYCLE  COST  IMPACTS  AND  TRADEOFFS 


SYSTEM  EMPLOYMENT  CONCEPT 


CAT-ASVAB  PROGRAM  CONCEPT  OF  OPERATION 
AND  COST/BENEFIT  ANALYSIS 


Increases  Applicant  Travel  Time  &  Costs 
Reduces  Recruiter  Travel 
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Increases  Applicant  Travel  Time  &  Costs 
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MEPS  Anticipate  Recruiter  Misuse  If  Not  Controlled 
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Best  Platform  for  Future  Test  Implementation 
Resolves  Many  Site  and  Security  Issues 
Potential  for  Multi-Use  Cost  Offset  (Mobile  Recruiting) 
Provides  DoD  Greater  Control  over  Testing  Resources 
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Within  Each  Service 

—  CAT-ASVAB  Administered  at  RTCs 

—  New  Computerized  Test  Battery  Administered  at  RTCs 
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Refine  and  Validate  CAST  for  Use  as  DoD-Wide  Preenlistment 
Screen 


Topical  Area  Review  Attendees  -  8  &  9  December  1988 


Name 

Address 

Teleohone 

ALLUISI,  Earl  A.. 

OUSD(A)/DDDR&E(R&AT) 

The  Pentagon,  Room  3D  129 
Washington,  DC  20301-3080 

AV  225-9777 

ARABIAN,  Jane 

ARI 

PERI-RS 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 

703-274-8275 

BLOXOM,  Bruce 

DMDC 

Monterey,  CA  93940 

AV  878-2951 

BRANCH,  Richard 

HQ  USMEPCOM 

Testing  Directorate 

2500  Green  Bay  Road 

North  Chicago,  IL  60064 

312-688-3438 

CARROLL,  Robert  M. 

CNO  (OP-01B2) 

Washington,  DC  20350 

AV  224-5618 

COOK,  Paul  Lt.Col. 

HQ  USAF/DPXOA 

The  Pentagon 

Washington,  DC  20330 

202-695-9855 

EATON,  Newell  Kent 

ARI 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 

AV  284-8844 

HADDAD,  Genevieve 

HQ  AFSC/XT 

Andrews  AFB,  MD  20334-5000 

AV  858-2366 

HANSER,  Larry 

ARI 

5001  Eisenhower  Avenue 
Alexandria,  V A  22333-5600 

AV  284-8275 

HIGGINS,  Mike  Lt.  Col. 

USAF  SAF/AQT 

The  Pentagon 

Washington,  DC  20330 

202-695-7866 

JEWEL,  James  S.  Col. 

HQDA  (DAPE-MPA) 

The  Pentagon 

Washington,  DC  20310 

AV  227-331 8 

JOHNSON,  Edgar 

ARI 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 

AV  284-8636 
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KLEMMER,  Robert 

HQDA 

Research  &  Studies  Office 

The  Pentagon  -  Room  2D736 
Washington,  DC  20330 

202-695-0516 

KUHN,  Bart 

CNO  (OP-987H) 

Rm  5E683  Pentagon 

Washington,  DC  20350-2000 

AV  224-4480 
202-694-4480 

KYLLONEN,  Patrick 

AFHRL/MOEL 

Brooks  AFB,  TX  78235 

AV  240-3570 
512-536-3570 

LANCASTER,  Anita 

OASD  (FM&P) 

The  Pentagon  -  Room  2  B  271 
Washington,  DC  20301 

AV  227-9271 

LANTERMAN,  Richard 

US  Coast  Guard 

G-PWP 

2100  2nd  Street,  S.W. 

Washington,  DC  20593-0001 

202-267-2986 

LEHNUS,  Jerry 

Defense  Manpower  Data  Center 
1600  Wilson  Boulevard,  Suite  400 
Arlington,  VA  22209 

703-696-4066 

LEIGHTON,  Dan  LTC,  USAF 

AFHRIVMO 

Brooks  AFB,  TX  78235 

AV  240-2244 

MARTIN,  Clessen 

CNO  (OP- 136) 

Navy  Annex 

Wing  8,  Room  2840 

Washington,  DC  20350 

A  V  224-5559 

MOSES,  Frank 

ARI 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 

AV  284-8816 

OLSON,  Darlene 

ARI/PERI-RS 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 

AV  284-8275 

ORLANSKY,  Jesse 

IDA 

1801  N.  Beauregard  Street 
Alexandria,  VA  22311 

703-578-2836 

PASS,  John 

NPRDC/12 

San  Diego,  CA  92152 

AV  553-7642 

PATSY,  William  (Ron) 

HQDA  (DAPE-MPA) 

The  Pentagon  -  Room  2  B  717 
Washington,  DC  20310 

AV  225-0836 
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REE,  Malcolm 


AFHRL/MOA 
Brooks  AFB.TX  78235 


AV  240-3256 


(  RIEMER,  Stephen 

RUCK,  Henk 

1  RUMSEY,  Michael 

SANDS,  William  A. 

> 

SELLMAN,  W.  Steven 

(  SENAIKO,  H.  Wallace 

TANGNEY,  John 
'  WALKER,  Clinton 

) 

> 

) 

) 


AF  Human  Systems  Division 
(HSO/CIF) 

Brooks  AFB.TX  78235 

AV  240-3687 

HRIVCA(Acting) 

Brooks  AFB.TX  78235-5601 

AV  240-3605 

ARI 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 

Av  284-8275 

NPRDC  -  Code  13 

San  Diego,  CA  92152-6800 

AV  553-9266 
619-553-9266 

OASD(FM&P) 

The  Pentagon  -  Room  2  B  27 1 
Washington,  DC  20301 

AV  225-5525 

Smithsonian  Institution 

801  N.  Pitt  Street 

Alexandria,  V A  22314 

703-357-1829 

AFOSR/NL 

Washington,  DC  20332 

202-767-5021 

ART  (PERI-RS) 

5001  Eisenhower  Avenue 
Alexandria,  V A  22333-5600 

703-274-8275 
AV  284-8275 
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OFFICE  OF  THE  SECRETARY  OF  DEFENSE 

WASHINGTON,  D.C.  20301  ®  ®  NOV  1988 


MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  THE  ARMY  ( M&RA) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (M&RA) 
ASSISTANT  SECRETARY  OF  THE  AIR  FORCE  (AQ) 

SUBJECT:  Topical  Area  Review  --  Testing  R&D  and  Planned 

Applications  to  Enlisted  Personnel  Selection  and 
Classification 


A  topical  area  review  of  testing  R&D  and  planned  applica¬ 
tions  to  enlisted  personnel  selection  and  classification  will  be 
held  on  December  8-9,  1988,  at  the  Institute  for  Defense  Analyses 
(IDA),  Alexandria,  Virginia.  The  review  is  intended  to  bring  the 
testing  R&D  and  user  communities  together  to  ensure  that  each  is 
aware  of  developments  and  plans  of  the  other,  and  that  the  R&D  is 
relevant  to  existing  and  planned  programs. 

Interest  and  concern  have  been  expressed  by  the  Congress, 
OSD,  and  the  Services  regarding  the  R&D  test-development  process 
to  ensure  equitable  screening,  selection,  and  classi f ication  of 
enlisted  personnel.  A  major  purpose  of  this  review  is  to  cover 
the  R&D  being  conducted  and  planned  in  these  areas,  as  well  as 
plans  for  the  implementation  of  the  products  being  developed  by 
the  R&D.  Additionally,  the  review  will  provide  an  opportunity  to 
identify  any  caps  in  R&D  that  need  to  be  filled,  as  well  as  any 
new  developments  or  initiatives  for  which  transition  plans  need 
to  be  drawn. 


Policy  perspectives  will  be  provided  at  the  review  by  the 
Assistant  for  Training  and  Personnel  Systems  Technology  (ODDDR&E/ 
R&AT )  and  the  Director  for  Accession  Policy  (OASD/FM&P).  We 
request  that  you  nominate  one  or  more  persons  from  each  of  the 
R&D  and  user  sides  of  the  enlisted  personnel  testing  communities 
to  represent  your  Military  Department  or  Service(s)  at  this 
meeting.  The  tentative  agenda  is  attached.  Topics  to  be 
addressed  are  to  include  both  the  existing  and  planned  R&D  and 
use  of  tests  for  enlisted  personnel  selection  and  classification, 


and  should  include  present  R&D  program  status,  accomplishments, 
future  plans,  and  5-year  funding  profiles.  Please  provide  by 
December  1,  1988,  the  name(s)  of  your  R&D  representatives ( s)  to 
Dr.  Earl  A.  Alluisi  at  (202)  695-9777,  and  your  user  represen¬ 
tative^)  to  Dr.  W.  S.  Sellman  at  (202)  695*-5525. 


/  George  P.  Millburn  /^ffyaiTrC a.  Green, 

Deputy  Director  Assistant  Secretary  / 

Defense  Research  and  Engineering  (Force  Management  an4 
(Research  and  Advanced  Technology) 


>f/Def  pnse 
“l^ersonnel ) 


Attachment: 
As  stated 
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